With bandwidths and data transfer rates of telecommunication systems continuing to advance, many of today's computing needs are shifting towards network-based distributed computing systems, commonly referred to as “cloud computing.” Generally speaking, cloud computing involves offloading certain computing tasks to one or more remote computing resources and utilizing the remote computing resources to provide services to a local device. Cloud computing is often implemented as a client-server model, with the remote server providing desired computing capabilities to a local client device, avoiding the need for the local client device to perform certain computing tasks on its own. In many instances, the computing resources can be entirely or almost entirely offloaded to the server, with the client providing little or no more than a user interface for the cloud computing task, allowing potentially any internet connected device to serve as a client for a particular application.
Cloud computing has been adopted in a wide variety of computing services, including remote data storage solutions, music and video streaming services, and a wide variety of other consumer and business solutions. In general, cloud computing provides several well-known benefits, including increased efficiencies through the use of shared resources, increased flexibility in accessing content without restriction to a particular device, lower up-front costs when hardware upgrades become available, and other advantages.
One arena in which cloud computing has yet to achieve widespread adoption is in the ever popular field of video gaming. Video games, including personal computer (PC) games, home console games, handheld console games, and the like, remain a popular source of entertainment among consumers. With a large and ever growing collection of available video game titles, cloud gaming provides a particularly attractive solution to many video game demands. Ideally, within a cloud gaming framework a user would be able to instantly access any video game title from among a vast collection of remotely stored titles, without needing a physical copy of the game, without having to wait for the title to download, and without locally possessing the hardware system or resources needed to run the game. Rendering of graphics, changing game state in response to user inputs, and other gaming computations could then be performed remotely in the “cloud” and sent to the user's local device. Unfortunately, implementing this type of service within a traditional cloud computing server design presents several technical challenges that have thus far prevented widespread adoption of cloud gaming for many video game platforms.
One challenge is that many of today's video games are very fast paced and action packed, with advanced graphics that require a significant amount of computing resources to render. Particularly in a video streaming based server design, in which all of the game computations and graphics rendering computations are performed remotely, the game needs be able to perform all these tasks and deliver a compressed audio/video stream to the client device with minimal to no perceptible latency in response to control inputs. Otherwise, this may disrupt the timing for the player and fail to replicate the responsiveness of a traditional locally executed gaming session.
Another challenge stems from the fact that there has traditionally been a distinction between PC games on the one hand, which are designed to run on personal computers, laptops, and other computing systems of a general purpose nature, and console games on the other hand, which are designed to run on specialized systems known as video game consoles. Video game consoles are typically designed as embedded platforms having unique computing architectures specially adapted for the video gaming environment. Using dedicated console hardware as a video gaming platform provides several benefits, particularly in the form of optimizing the hardware for the high computing resource demands of graphics rendering and gaming computations required to produce many of today's video games. As a result, while PC games have achieved some popularity, console games have traditionally dominated the video game market and may continue to do so in the future.
However, adapting video game consoles and other embedded platforms to a cloud streaming service presents several challenges. Extensive modifications of the hardware or software architecture of the specialized platform may present compatibility issues with applications designed for the specialized platform. As a result, the desired application may need to be run on directly on the compatible architecture on a separate “terminal system” at the server side in order to execute the application for streaming over the network. The terminal system may be a game console architecture or other existing embedded platform designed to run the specialized application.
In order to stream the application over the network, techniques are needed to capture video frames from the application running on the terminal system in order to compress the output video signal before transmission. Conventional video capture cards are capable of capturing video frames transferred through a video interface link, such as HDMI, DVI, VGA, and the like, so that they may be compressed and sent to a remotely located display. However, delay in the time it takes to transfer each frame to the capture device contributes to latency between the time the frames are generated by application content and the time each frame is presented on a display at the client side. Perceptible latency may result from a combination of video capture delay, compression delay, network delay, and other sources. In cloud gaming and other cloud computing environments, minimizing latency is a critical task, and every millisecond counts in providing a responsive experience for the user at the client side.
It is within this context that the present disclosure arises.